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ABSTRACT 


This thesis proposes new methods, based on dynamic programming, for solving 
certain single-stage and multi-stage integer stochastic knapsack problems. These 
problems model stochastic portfolio-optimization problems (SPOPs) which assume 
deterministic unit weight, and normally distributed unit return with known mean 
and variance for each item type. Given an initial wealth, the objective is to select 
a portfolio that maximizes the probability of achieving or exceeding a specified final 
return threshold; the multi-stage problem allows revisions of the portfolio at regular 
time intervals. An exact method is developed to solve a single-stage SPOP with 
independence of returns among item types. For a problem from the literature with 
11 item types, this method obtains an optimal solution in a fraction of a second on 
a laptop computer. An approximation method, based on discretization of possible 
wealth values, is developed to solve a multi-stage SPOP with inter- and intra-stage 
independence of returns among item types. Running on a desktop computer, this 
approximation method solves a 3-stage problem with 6 item types in under 12 minutes. 
With finer discretization in a 3-stage problem with 8 item types, the solution time is 
about 46 minutes. 
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DISCLAIMER 


The reader is cautioned that computer programs developed in this research 
may not have been exercised for all cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and 
logic errors, they cannot be considered validated. Any application of these programs 
without additional verification is at risk of the user. 
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LIST OF SYMBOLS 


Symbols in bold font are the vector form of the variables or parameters rep¬ 
resented by the corresponding symbols in regular font. For example, r is the vector 
form of r. 

The index of an element in a vector is represented as a subscript. The variables, 
parameters and solution sets related to time t in a multi-stage problem are labeled 
with the subscript t; for elements of a vector, t is the index after the element index. 
For example, Xkt is the kth. element of vector x at time t Other subscripts and 
superscripts, which can be easily understood by the context in which they exist, are 
noted in the following list: 

_ “ Lower and upper bounds, respectively, of the argument 
' " Intermediate or non-optimal representations of a variable or parameter 
* superscript indicating optimal solution of a problem 

+ Subscript index of positive integer set; superscript index of an optimal objec¬ 
tive value 

— Subscript index of negative integer set; superscript index of an optimal objec¬ 
tive value 

a Abitrary constant 

c Desired minimum total return, or total return threshold 
det Determinant of a matrix 
E Mathematical expectation operator 

/ Function (optimal value function in a dynamic programming procedure) 
h Probability density function 

i Subscript index for total mean return 

I Set of total mean in returns 

j Subscript index for total variance in return 
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J Set of total variance in returns 
k Subscript index of item type 

K Number of item types; as a superscript, the dimension of a set 
K Set of candidate portfolio items 

I Index of an item among its type; quantity of an item type 
L Maximum quantity affordable for an item type 
£ Set of affordable quantities of items 
min Subscript for a minimum value 
n Superscript index for level of wealth realization 
N Normal distribution; number of possible wealth realization levels 
P Probability function 

p Probability 

r Unit return of an item 

t Subscript index for time 

T Length of planning horizon for a multi-stage problem; as a superscript, the 
transpose of a vector; as a subscript, the end of a planning horizon 

U Utility value or function 

U Set of values for total mean return 

V Total or unit variance in return 

V Set of values for total variance in return 

w Weight or cost of an item 

W Capacity of knapsack; available wealth for investment 
X Quantity of items, which are of the same type, to include in the portfolio 

X Solution set of a portfolio decision 

y Decision (indicator) variable 

Z The set of integers 
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Z Standard normal quantile 

fj, Total or unit mean return 

$ Standard normal cumulative distribution 

p Objective function value 

C Objective function value 
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EXECUTIVE SUMMARY 


This thesis develops new methods for solving certain probabilistic versions of 
portfolio selection problems, in short, “stochastic portfolio-optimization problems.” 
In these problems, each item type of the portfolio has deterministic unit cost, but 
probabilistic unit return value with known probability distribution. We assume a 
normal distribution in our study. Given an initial wealth, an investor would like to 
determine a portfolio with the best probability of achieving or exceeding a specified 
return threshold. The stochastic portfolio-optimization problems are closely related to 
“stochastic resource-allocation problems” which expend limited resources to acquire 
a system with maximized expected utility. Hence, the proposed solution techniques, 
with modifications, have wide applications in resource-allocation problems such as 
cargo loading, capital budgeting, project selection and weapons-mix problems. 

The problems considered in this thesis assume that the returns for aU item 
types are independent of each other. The first problem involves selection of a portfolio 
which cannot be altered until the end of an investment period when the portfolio is 
cashed; hence, it is called a “single-stage stochastic portfolio-optimization problem.” 
We develop a method that provides an exact solution to this single-stage problem. 
This method examines each item type one by one, and considers different possible 
mixes of item-type quantities to yield specific mean and variance pairs for returns. 
Hence, the problem is decomposed into smaller and more manageable problems. At 
the examination of the last item type, an optimal portfolio, which has the highest 
probability of achieving or exceeding the return threshold, is selected. This stage- 
by-stage solution approach is a classical example of dynamic programming. For a 
problem from the literature with 11 item types, this method obtains an optimal 
solution in a fraction of a second on a laptop computer. 

The problem just described involves a one-time decision. In practice, portfolios 
are revised at regular time intervals during the planning horizon. These stochastic 
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problems need decisions based on sequences of outcomes revealed over time; hence, 
these are multi-stage problems. We also consider, in this thesis, a multi-stage stochas¬ 
tic portfolio-optimization problem which assumes that the returns for all item types 
are independent of each other not only at a specific point in time, but also across 
time. An approximation method, based on an extension of the classical dynamic¬ 
programming technique, is developed for this multi-stage problem. Running on a 
desktop computer, this approximation method solves a 3-stage problem with 6 item 
types in under 12 minutes. A more precise approximate solution is obtained for a 
3-stage problem with 8 item t 3 q)es in about 46 minutes. A possible improvement to 
our approximation method through the use of a sampling approach is also suggested 
at the end of the thesis. 

In this thesis, we have shown the relevance and efiiciency of dynamic-program¬ 
ming approaches to solving single- and multi-stage stochastic portfolio-optimization 
problems. Moreover, our multi-stage method, with modifications, has the potential 
to handle problems with certain dependencies among the item types. These de¬ 
pendencies add more realism to our problems, and extend them to possible future 
developmental work. 
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I. 


INTRODUCTION 


This thesis proposes new methods, based on dynamic programming (DP), for 
solving some stochastic variants of the classical general integer knapsack problem. In 
particular, we consider single- and multi-stage stochastic portfolio-optimization prob¬ 
lems (SPOPs) which assume deterministic unit weight, and normally distributed unit 
return with known mean and variance for each item type. Given an initial wealth, 
the objective is to select a portfolio that maximizes the probability of achieving or 
exceeding a final return threshold. We develop solutions for a single-stage (single¬ 
period) integer SPOP with independence of returns among the various item types in 
the portfolio, and a multi-stage (multi-period) integer SPOP with inter- and intra¬ 
period independence among item types. These problems, along with associated so¬ 
lution techniques, relate closely to single-stage stochastic knapsack problems, single- 
and multi-stage portfolio-optimization problems, multi-stage stochastic programs and 
stochastic dynamic programs. These problems are also related to stochastic resource- 
allocation problems. Hence, the proposed solution techniques, with modifications, 
have wide applications in problems such as cargo loading, capital budgeting, project 
selection and weapons-mix problems. 

A. BACKGROUND 

This section provides some background on the single- and multi-stage stochas¬ 
tic knapsack problems (SKPs) which we solve in this thesis. 

1. Single-Stage Stochastic Knapsack Problems 

In the classical knapsack problem, a hiker wishes to determine which of a set 

of items to carry on a backpacking trip. Each item has a weight and a “return value” 

to the hiker. Given a knapsack with limited weight capacity, the hiker wishes to 
♦ 

determine the load to pack, so as to achieve the most valuable total return. This 
problem is also known as the flyaway kit problem (Taha 1992, pp. 358). It becomes 
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a general integer problem when the hiker can pack more than one unit of a particular 
item type. 

In a stochastic variation of the knapsack problem, the returns from each item 
are random with known distributions. Random returns are common in the real world. 
For example, the returns of different financial instruments such as stocks and bonds, 
the revenue from a production plant or project, and the damage on a target resulting 
from a missile attack are all random in nature. 

The integer SKP that we study here may be described as follows: 

Indices 

k item type. A: € /C = {1,2,..., K} 

I index of an item among its own type, I € {1,2,..., Lk} 

Data 

rki return for item I of type k] the marginal distribution of is assumed to be 
normal with mean Hk and variance Vk, i.e., rki ~ N{^lk, Vk) V k, 1; 
r — (r"ll, ^12) • • •) '^ILi ) ^21) • • • ) “^KEk) 

(Different variants of SKP arise with different distributions for and dependency 
structure among the r«.) 

Wk deterministic weight of each type-fc item, WkE Z+Vk 
c desired minimum total return, i.e., the total return threshold 
W capacity of the knapsack, W e Z+ 

Decision Variables 

yki 1 if item I of type k is included in the knapsack; else 0 
y — (i/ii) 2 /i 2 ) • • •) Vihi ) 2 / 21 ) • • •) yKLji) 
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Formulation 


GSKP(IF) 


max 

y 


( K Lk 

Y^^rkiVki 

k=l 1=1 



S.t. 


Y^Ylwkyki < W 


k=l 1=1 


Vki e {0,i}'ike}C, 1 = 1,2,...,Lk. 


Here, Eti Vid is the number of items of type k to be included in the knapsack. 
The deterministic weight of each type k item is a positive integer Wk and W is the 
known, integer capacity of the knapsack. The returns r^i, ..., rkik ^ specific item 
type k are identically and normally distributed with mean fik and variance Vk. In 
some situations, this distributional form is reasonable, and furthermore, it leads to 
computationally tractable models. 

The problem in GSKP(VF) is to select an optimal y that maximizes the prob¬ 
ability that the total return r^y meets or exceeds threshold c. Depending on the 
specific problem, there might be dependency in the imit return among items of the 
same type. In this thesis, we describe the extent of dependency among items of 
the same type with “complete independence,” “partial dependence” and “complete 
dependence within item types.” Similarly, for zero, partial or perfect dependency 
among the return from items of different types, we use the descriptions of “complete 
independence,” “partial dependence” and “complete dependence among item types,” 
respectively. 

Carraway, Schmidt and Weatherford (1993) use “generalized dynamic pro¬ 
gramming” developed by Carraway, Morin and Moskowitz (1989) to solve GSKP(W) 
when there is complete independence among and within all item types. The more 
efficient techniques of Morton and Wood (1998) are applicable not only to this com¬ 
pletely independent case, but also (with modifications) to situations where returns 
are identical within an item type, i.e., when there is complete dependence within item 
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types. In the latter case, the SKP corresponds to investment in multiple financial in¬ 
struments such as stocks and bonds. The cost of each share of stock k is Wk dollars, 
the total wealth available for investment is W dollars, and the return from every share 
of stock k is Tk, which is a normal random variable. The objective is to invest limited 
assets so as to maximize the probability of achieving or exceeding a specified return. 
Hence, this model may be viewed as a simple, single-period SPOP; we develop a DP 
method for solving this problem in this thesis. 

The portfolio-optimization version of the model GSKP(kP) may be simplified 
to: 

SKPO(H^) 

(K 

max P y) TkXk > 

^ \k=i 

K 

s.t. ^WkXk < W 

A:=l 

Xk ^ V fc E /C 

where r*, = r^i = = • • • = Vki^i w.p.l, and Xk is the number of type-A: items to 

include in the portfolio. 

The objective function expresses a variant of the “safety-first criterion” (Pyle 
and Turnovsky 1970). For this criterion, a disaster level of returns is first specified; 
the objective is to minimize the probability that the actual total return is worse (less) 
than or equal to the disaster level. The objective in SKPO(W) is essentially equivalent 
to this criterion. 

2. Multi-Stage Stochastic Knapsack Problems 

A single-stage problem involves a “one-time” decision. In practical planning 
problems such as production scheduling and power capacity expansion, it is not un¬ 
common to find that multiple decisions are required at different phases of the planning 
horizon. These problems need decisions based on sequences of outcomes revealed over 
time (Birge and Louveaux 1997, pp. 128). 
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Consider a multi-stage problem expanded from the single-stage SKPO(W). 
The scenarios are standard to most multi-stage portfolio models: We begin with 
fixed capital, we invest in a set of financial instruments, we review and modify our 
investment portfolio after certain period of time has elapsed, and repeat this for a 
specified number of time periods. Our objective is to maximize the probability that 
we achieve or exceed a final target for total wealth (although other objectives are 
possible). We assume that no fee is charged for any transaction and no money is 
borrowed for investment. We are, therefore, only concerned about the uncertainty 
in the returns of the financial instruments. In addition, at the end of each time 
period, the portfolio rebalancing decision is only restricted by the available wealth 
accumulated from investments in the previous periods. The exact problem is defined 
in Chapter III. 

The multi-stage SKP described above might be handled using the methods 
developed for the more general “multi-stage stochastic programming problems.” Birge 
and Louveaux (1997, pp. 233-252) describe some of the exact methods that have been 
implemented with some success. Because exact methods are restricted to solving 
problems of moderate size and complexity, approximations (with deterministic or 
probabilistic bounds on accuracy) are often used to provide good solutions. Birge 
and Louveaux (1997, pp. 353-370) also discuss some of these techniques. 

Much recent work on multi-stage stochastic programming involves developing 
models for financial planning problems faced by investment firms and corporations 
with large portfolios, e.g., insurance companies (Carino et al. 1994). Such efforts 
include Klaassen’s (1998) use of a state and time aggregation method in stochastic 
programming models for asset/liability management, and Hiller and Eckstein’s (1993) 
use of massively parallel Benders decomposition in solving a stochastic portfolio model 
for fixed income asset/liability management. These models consider more complex 
economic factors than our problem, such as the uncertainty in interest rates and the 
need for liability management resulting from loans made. Hiller and Eckstein (1993) 
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adopt an efficient frontier approach (Sharpe 1970, pp. 52): this provides the portfolio 
manager with a set of portfolios that are efficient with respect to risk and return, 
rather than a portfolio that is optimal with respect to a single composite criterion. 
None of the above approaches fits well into our problem of determining an optimal 
initial portfolio decision based only on uncertain returns. 

Stochastic dynamic programming (SDP) is a natural choice for our multi- stage 
SPOP. It is similar to deterministic dynamic programming except that decisions at 
a particular stage t depend on the realizations that have occured up to that point 
in time. A short discussion of SDP can be found in Kali and Wallace (1994, pp. 
124-129). 

B. SCOPE 

Morton and Wood (1998) state that their DP algorithm for the independent 
case may be extended to the dependent case, i.e., when there is complete dependence 
within item types. But those authors do not provide details. In Chapter II, we 
establish a variant of Morton and Wood’s DP algorithm to handle the portfolio- 
optimization model SKPO(W) with complete dependence within item t 3 ^es, and 
complete independence among item types. In Chapter III, the model SKPO(W) is 
expanded to a sequential multi-stage decision problem. We then a new DP-like algo¬ 
rithm for solving this multi-stage portfolio-optimization model with complete inter- 
and intra-stage independence among the item types. Finally, we conclude this thesis 
and propose some future developmental work in Chapter IV. 
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II. A SINGLE-STAGE STOCHASTIC 
KNAPSACK PROBLEM 
(SSKP) 


In this chapter, we present a dynamic-programming solution to the single-stage 
portfolio-optimization model SKPO(W) with complete dependence within each item 
type as well as complete independence among item types. This solution is a variation 
of Morton and Wood’s dynamic programming method (1998), which assumes com¬ 
plete independence within each item type. Hence, much of this chapter parallels that 
paper. 


A. MATHEMATICAL FORMULATION OF SSKP 

The problem which we wish to solve is re-stated for clarity as follows; 


Indices 


k item type, k E IC = {1,2,..., K} 

Data 

Tk return for each item of type fc; the marginal distribution of r*, is assumed to 
be normal with mean fXk and variance Vk, i.e., rk ~ N{nk, Vk) V k; 
r = {ri,r 2 ,...,rKY 

Wk deterministic cost of each item of type k, Wk E Z+\/ k 

c desired m inimum total return, i.e., the total return threshold 

W initial total wealth, W E Z+ 

Decision Variables 

Xk number of type-fc items to include in the portfolio, -x. = {xi, X 2 ,..., xkY 
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Formulation 


SKPO(TF) 

(K 

P TkXk > c 

^ \k=i 

K 

s.t. ^WkXk < w 

fc=l 

Xk € Z+\/ k etc . 

B. ASSUMPTIONS AND REFORMULATION 

For the purpose of model formulation, we make the following assumptions: 

1. The mean of all item types can be integerized through scaling and rounding 
with little loss of accuracy, when necessary. Therefore, we assume that fik € 
Z+\/keK. 

2. The returns are independent among item types, i.e., Uat = 0 V a, 6 G /C, a ^b, 
where Vab denotes the covariance between the returns of item types a and b. 

3. There is one and only one riskless item for the portfolio. A riskless item, 
classified to be type A: = 1 throughout this thesis, has return ri ~ A'(a(i, 0) 
where /Zi >W]_. 

4. The initial wealth W > minfcgx: Wk^ this simply implies that the optimal knap¬ 
sack will not be empty. 

5. The threshold c is greater than the total return from a portfolio of riskless 
items, i.e., c > ii-\.\W/wi\ where [aj denotes the largest integer not exceeding 
a. Hence, the threshold c cannot be achieved with probability one. 

6. There exists at least one feasible solution x with positive total variance. Thus, 
P (E^i fkXk > c) > 0. 

The last three assumptions allow us to focus on portfolios with positive variance; 
portfolios with only riskless items, i.e., zero variance, are handled as special cases. 

We now reformulate SKPO(W) using an equivalent deterministic objective 
function. For compact representation of the problem, the following vector notation is 



unit returns, r = (ri, r 2 ,..., vkY, 

portfolio, X = XkY, 

unit cost, w = {wi,W 2 ,..., wkY, 

unit mean returns, fj, = (/ii, , IJ-kY, and 

unit variance in returns, v= (ui, t; 2 ,..., 


Since Vk ~ N{iik, Vk), the total return of portfolio x is given by 
Y.k&Kf'kXk = r^x ~ iV(^^x, v^x^) 

where x^ = {x^,xl,...,x^^). Therefore, we may convert the problem in SKPO(W) 
to a non-linear optimization problem. With the implicit constraint of positive total 
variance, i.e., v^x^ > 0, problem SKPO(W) may be reformulated as 


SKPl(W) 


P* (^) = /v^v^x^ 

s.t. w^x < W 


X € Z^. 


The deterministic objective function is valid because 


■y/v^x^ 


= P iV(0,l)> 


v- 


y/r-^2 


c — 


yA 


rTy.2 ) 


(11.1) 


given v^x^ > 0, and therefore, the probability of achieving the return threshold c 
is maximized by minimizing the right-hand side of the inequality. This gives the 
objective function of SKPl(W). 


C. DYNAMIC PROGRAMMING FOR SSKP 

In this section, we present a DP algorithm DSSKP to solve SKPl(W) or, 
equivalently, SKPO(W’). 
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1. Concept 

Suppose that ji and p, are, respectively, valid lower and upper bounds on 
where iC{W) is an optimal solution to SKPl(Vr). Furthermore, assume 
that p is tight, i.e., p = IJ>*{W) where p*(W) is determined by solving a simple 
deterministic knapsack problem: 

KP(VF) 

ff{W) = m^ 

s.t. < W 

X € Z^. 

In this thesis, we set = min^gx: Pk^ For problems of larger scale, p can be set tighter. 
Now, define U = {p, p + I,p} and re-arrange SKPl(TF) to obtain 

SKPla(TF) 

p*{W) = min min (c — p)/V^^ 

s.t. piF-x. = p 
w^x < W 

X € Z^. 

As noted by Morton and Wood (1998), for fixed p < c, p*{W) is minimized 
when v^x^ is maximized, and for fixed p> c, p*(W) is minimized when v^x^ is mini¬ 
mized. = c is a special case in which the objective value is zero. Consequently, there 
are three cases to consider for solving SKPl(W) based on the p =p*{W) determined 
by KP(W). 

a. Case 1, p < c 

In this case, for each pEU,we first solve 
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MAX(W;)U) 


C^(//) = max 

S.t. /X^X = fi 

w^x < W 
y. e 

for optimal solution x'(/i). For any // such that MAX(W,/li) is infeasible, we define 
Then, the optimal objective value of SKPla(W’) is 

P* (^) = w \/v^x'2(^). 

Hence, any solution in (J {x'(/Li)} which satisfies (c — /x)/-^v^x'2(/x) = p*(W) is an 

y>£U 

optimal solution to SKPla(W) and therefore SKPl(W). It can be reasoned from 
(II.l) and the objective function of SKPla(lF) that, in this case, 

0 < p*{W) < oo; 


hence. 


0 < P (r^x*(lF) > c) < 0.5. 


h. Case 2, p, = c 

Since ft is a tight upper bound, there exists a feasible solution x'(c) 
to SKPla(W). Let x'(c) denote the solutions to KP(W) that achieved /x^x'(c) = c. 
This is an optimal solution to SKPla(W) since it yields an objective value of zero 
which is higher than any of the objective values in Case 1 (/x < c). Hence, 


P*W = 0, 


and 

P (r^x*(IF) > c) = 0.5. 

Note that the total variance of x*(IF) is irrelevant in this case. We 
ignore variance and take any optimal solution of KP(W). 
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c. Case 3, Ji> c 

In this case, we first define the lower bound on /r as = max{^, fc]} 
(where [-1 is the ceiling operator that returns the smallest integer not less than its 
argument). Then, we solve 




C-(fi) = inin 

s.t. = /X 

w^x < W 

X E Zf 

for x'(/u), for each jx eW — {y!,!!! + 1,...,/x}. Similar to MAX(W,/x), we define 
^“(/x) = 00 for any /x such that MIN(Vr, /x) is infeasible. Then, the optimal objective 
value of SKPla(W) is 

p*{W)^ mm (c-/x)/^v^x'2 (//)). 

Hence, any solution in [J {x'(/x)} which satisfies (c — /x)/^v^x'2(^)) = p*{W) is an 

II&4' 

optimal solution to SKPla(W) and therefore SKPl(W). In this case. 


—00 < p*{W) < 0 
4^ 0.5 < P (r^x*(VP) > c) < 1. 


2. Algorithm Details 

We use a three-phase algorithm to solve SKPl(W): Phase 1 solves KP(W) to 
determine which of Cases 1, 2 or 3 to consider for solving the problem. If Cases 1 
or 3 are considered, Phase 2 solves a series of MAX(W,/x) or MIN(W,/u) problems. 
Phase 3 extracts and prints the optimal solutions determined. 
a. Phase 1 

Phase 1 solves KP(W), a standard knapsack problem, using the follow¬ 
ing basic DP formulation (Dreyfus and Law 1977, pp. 108-110): 
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DP Recursion for solving KP(W) 

Optimal Value Function 

lj,*{w) — maximum total mean return attainable with available wealth w. 
Recurrence Relation for w = lymin,Wmmwhere Wrmn — minfcgx: Wk- 
= maXk^K {fji*{w-Wk) + fik}- 
Boundary Conditions 




—00, w < 0 

0, 0 ^ W ^ '^min I' 


Answer 


Now, if Phase 1 determines that /x*(W) = c, the algorithm will skip to Phase 3 
where an optimal solution is extracted. Otherwise, Phase 2 is run to determine the 
maximum variance in returns by solving MAX(W, for all if /i*(W) < c, or 

the minimum variance in returns by solving MIN(VP, //) for all /i € W' if iU*(lV) > c. 

b. Phase 2 

Let MAXE(ty, (j) be the problem MAX(W, with the constraint w^x < 
W replaced by w^x = w. In this phase, MAX(kF, fi) for each fj, eU are solved by 
first determining the solutions to MAXE(w,^) for w E {w,w +'^1, ... ,W} where 
w = miufcexWfc- After that, a solution that yields the lowest objective value of 
SKPl(W) is selected. In the same way, solutions of MIN(kP,/x) for each fx eW are 
determined by solving a series of problems MINE(t(;, fi) which is defined analogously 
to MAXE(w, //). 

The standard DP algorithm for the simple KP is extended to solved the 
two-constraint integer programs (IPs) MAXE(ty,/z). In each stage of the algorithm. 
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only one item type is considered for knapsack loading. Hence, the index of the item 
types, k, is used as the stage number. The DP formulation is: 

DP Recursion for solving MAXE(kF, 

Optimal Value Function ' 

fk{w\ /u') = maximum total variance in return from investing in item types 
0, 1, ..., fc given that wealth w' has been invested and a total 
return of n' is expected. Item type 0 is a dummy item type 
for boundary definition. 

Recurrence Relation for fc = 1, 2,..., If, w' = Wrmn, + 1,. • •»A^niin+ 
1,..., // where = min^ex: Wfc, ^min = min^exAffc 

/fe(w', ii') = maxa;fce{0. xk}{fk-i{w' - WkXk, n' - fikXk) + Vkxl} 

where Xk = min{[w7'“'fej> La^V/^^J}- 
Boundary Conditions 

foi.'^ j ) ~ ^ — ^min W ^ 0, fJ, ^ /^min 1) A^ ^ 

fk{0,0) = 0, k = l,2,...,K. 

Answer 

The above DP formulation may be modified to solve MINE(u;, /x): 


DP Recursion for solving MINE(kP, p) 

Optimal Value Function 

fk{w\ii') = minimum total variance in return firom investing in item types 
0, 1, ..., fc given that wealth w' has been invested and a total 
return of fx' is expected. Item type 0 is a dummy item type 
for boundary definition. 
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Recurrence Relation ior k = 1,2,K, w' — Wmin, Wm\n + l,...,w, //' = fimm, fhmn+ 

1.. .., where w = muifcgx: Wk, = min^gx: fJ^k 

fk{w', fJ,') = mma:fcg{0,...,Jfc}{/A:-l(w' - WfcXfc, n' - fikXk) + Vkxl} 
where Xk = min{[w7wfcj, 

Boundary Conditions 

yo(w^, = 00, v/ ^ Wmin Ij ^ Oj ^ /^min Ij 

/,(0,0) = 0, k = l,2,...,K. 

Answer 

fK{w, fJ,). 

Phase 2 is divided into Phases 2a and 2b. Using either of the above 
recursions, Phase 2a determines the values fkivj,^i) for k e K., w e {waan,Wiam + 

!>•••) S'nd fl G {//min) //min d" 1) • ■ • ) //}■• 

Subsequently, Phase 2b first redefines the lower bounds on n and w: if 
lJt,*{W) > c, y! = max {//, [c]}, and w' = argmin^={y,,...,wq{/i*(^y) > 4: else, //' = 
y and w' = ty. Now, define SKPE(i(;) as SKPl(lU) with the constraint w^x < W 
replaced by w^x = w. The optimal objective value p{w) of SKPE(w) is determined 
by examining all finite values of fk{w, y) computed in Phase 2a for each w G {w', uf -1- 

1.. ..,W}: 

where W"(w) = {y',y'+l, ... ,//*(ty)|C‘*‘(//) > —oo} for Case 1, or {y',y'+l,.,.,y*{w) 
|C“(//) < oo} for Case 3. The optimal objective value of SKPl(TU) is then 

p*{w)= min p{w). 
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c. Phase 3 

If Phase 1 determined that fi*{W) = c, this phase extracts the optimal 
solution to KP(W) as the optimal solution x*(W) to SKPl(W). Otherwise, the 
optimal solution is extracted as the best solution to SKPE(w) over all w € {w', w' + 
1 . W}. 

d. Model Refinements 

To capitalize on the computer runtime to produce additional useful 
results, the algorithm DSSKP also extracts the optimal solutions x*(w) to SKPl(u;) 
for all w e {w',w' + W} in the final phase. 

With insignificant extra effort, this additional information might be 
used to provide insight to a decision maker regarding the marginal loss of target- 
achievement probability due to loss in initial wealth. 

To enhance computational efficiency, the recursive computations of 
fk(w, jj) in Phase 2a and exhaustive search of p(w) in Phase 2b are made only in 
the range with p < p*(w) for each w. Because p*(w) is the maximum feasible total 
mean return given initial weight w, it follows that is invalid for values of p 

outside this range; the exhaustive search in Phase 2b need not examine these /’s. 

3. Algorithm DSSKP 

The DP algorithm for the SSKP (DSSKP) is presented as follows: 

Algorithm DSSKP 

Input: Data for SKPl(W) with K item types: integer vectors w > 0, ^ > 0, integer 
W > mmk^KWk, scalar c and real vector v > 0. 

Output (three possibilities): 

1. If maximum feasible p*(W) = c, optimal solution x*(W') and solution value 
p*(W) to SKP(W); 

2. if p*{W) < c, optimal solutions x*(tu) and solution values p*(w) to SKP(tu) 
for all w € {tu : minfcgx: Wk <w <W,w e Z+}] 
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3. if /i*(W’) > c, optimal solutions x*(ii;) and solution values p*{'w) to SKP(w) 
for all ty 6 {ty : ff{w) > c,w <W,w E Z+j; 

/* Phase 1 */ 

W ^ miiikeK'iOk; 

fi*(w) < - 00 V w with w — maxfc Wk <w <W\ 

0 V ty with 0<w <w — 1; 

For {w = ‘w to W) { 

k'{w) f- argmaXfcgx:{A^*(^w “ + f^k}; 

/J,*(w) ^ p*{w - Wk'(w)) + Mk'(w); 

} 

If {p*{W) = c) go to Phase 3; 

/* Phase 2a */ 

^<-minfcex:Aifc; 

'^unn millfc Wk } A^min fik j 

If < c){ 

fkiw, p) <-oo \/ k,w,fj, with 0 <k < K, Wmin — maxfc Wk<w< W, and 

Pmin - maxk Pk < P < 

fk{0, 0) 0 V A: with I <k < K; 

For (fc = 1 to iC and w = Wmin to W and p — pnaa to p*{w)) { 

Xk <r- min{[u;/tyfcj, [p/pk\}] 

^k{'^> y) ^ argniax^^g{o,...,Sfc}{/*-i(w - ^kXk, p - PkXk) + Vkxl\] 
fk{w,p) <*- fk-iiw - Wkxl{w,p),p - PkXkiw, p)) + Vkx'^iw,p); 

} 

} else /* if {p*{W) > c) */ { 

fkiw,p) <r- 4-00 ^ k,w,p with 0 <k < K, Wmm — max*: Wk <10 <W, and 

Pmin - maxfc pk<p<p; 

fk{0, 0) ^ 0 V A: with 1 < A: < iiT; 

For (A: = 1 to K and w = tymin to W and p = //min to p*{w)) { 
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Xk{w, n) ^ argniina.^g{o,..,xfc}{/fc-i(«' “ ^kXk, fJ^ - likXk) + Vkxl]; 
fk{w,n) ^ fk-i{w - Wkxliw, iJ,), fi - t^kx'liw, fi)) + Vkx'^{w, n); 

} 

} 

/* Phase 2b */ 

If > c) { 

y! -e- max{y, Tc]}; 

w' ^ argmax^=^,..._HA{//*(I^) > c}; 

} else { 

y' ■«—//; w' ^ w; 

} 

For {w = w' to W) { 

ifM, kt) *- axgmm^gz^//(^),fcgx(c - fk{w,y); 

p{w) ^ (c - yt)/yjfkt{w,yt)', 

1^{w) <- kt\ p!{w) <— yt; 
w{w) <r- argmin^jg{^/,...,^}P(wO; 

} 

/* Phase 3 */ 

If iy*{W) = c) { 

X •«— 0; w <—W; 

do { 

xi^iw) Xk'(w) + 1 ; 

W <r- W - Wk>(w)\ 

} while (m) 7 ^ 0); 

Print{“Phase 1 shows //*(”, W, “) = ”,c}; 

Print{“Solution to SKP(VF) for IF=”,IF,“is x*(IF) =”,x}; 
Print{“with optimal objective value p*(W) —’,p(l^)}; 
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} else { 

If > c) Print{“Phase 1 shows “) > ”,c}; 

else /* < c) */ Print{“Phase 1 shows /z*(”, W, “) < ”,c}; 

For {w = 'u/ to W) { 

X ^ 0; w w{w); 

p, k •«— A;'(w); 

do { 

Xk^x'l{w,p); 

W *-W — WkXk'i 

fj, < jj, flh^k) 

k k — 1; 

} while(w > 0); 

Print{“Solution to SKP(w) for w=”,w,“]s x*(w) =”,x}; 

Print{“with optimal objective value p*{w) =”,p{w'{w))}; 

} 

} 

} 

4. Computational Results 

For testing the algorithm, we use the same data set as Morton and Wood 
(1998), i.e., the data from Steinberg and Parks (1979), with the addition of a riskless 
item type. This item type is item type 1 which has Wi = 1, pi = I and Vi — 0. 
Morton and Wood (1998) solves SKPO(W) with the assumption of complete inde¬ 
pendence within item types, whereas our method solves it with the assumption of 
complete dependence within item types. Despite this, our method is an extension 
of Morton and Wood’s (1998) DP method (they denote it “DPSKP”). Hence, the 
relative computational time between their DPSKP and our DSSKP may provide a 
gauge for any extra complexity involved. The data are shown in Table I. 

We use W = 30 and c = 60 (instead of c = 30 in the two works mentioned) 
for testing. In our case, c = 30 is illogical as investing in all riskless items will result 
in a total returns of 30 with certainty and meet the return threshold with probability 


k 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

Wk 

1 

5 

7 

11 

9 

8 

4 

12 

10 

3 

6 

l^k 

1 

7 

12 

14 

13 

12 

5 

16 

11 

4 

7 

Vk 

0 

15 

20 

15 

10 

8 

20 

8 

15 

20 

25 


Table 1. Steinberg and Parks (1979) data with an additional riskless item. 

one. Morton and Wood (1998) programmed their algorithm in Turbo Pascal and 
ran it on a Dell Latitude XPi laptop computer with a 133 MHz Pentium processor 
and 40 megabytes of RAM. Our algorithm is programmed in Java 1.1.2 and run in 
Microsoft’s Windows 95 on a Dell Latitude LM laptop computer with a 166 MHz 
Pentium processor and 40 megabytes of RAM. 

For our problem, it is determined in Phase 1 that = 50 (i.e., /i*(W) < c) 

and w' = 1. Hence, we obtain solutions for the range of W = 1,2,..., 30. The total 
solution time, which includes the data input and solution printing, is 0.44 seconds. 
We compare this to the total solution time of 0.026 seconds reported in Morton 
and Wood (1998) (for all values of W between 3 and 30). Our solution time is an 
order of magnitude greater than theirs although both are less than a second. This 
difference may partly be due to the use of different programming languages and 
different computers. Cases of = c and > c have also been tested by 

setting W = 35 and 50 respectively. The corresponding total solution times for these 
two cases are approximately 0.06 seconds and 0.94 seconds. Given the restricted time 
frame within which this thesis must be completed, we state these results without 
further comparative testing. 

5. Comments 

Similar to Morton and Wood’s (1998) DPSKP algorithm, DSSKP is an exact 
method to solve an SPOP. Furthermore, it is simple to program. Based on the similar 
methodological extensions of basic DP algorithm as DPSKP, DSSKP can also be easily 
modified to accommodate bounded variables. Morton and Wood (1998) suggest that 


20 



this can be done by solving the bounded-variable version of SKPE(H^) which is just 
a two-constraint, bounded-variable knapsack problem. 

We have ignored cases in which v^x^ = 0 might be optimal. These special 
cases can be easily checked in Phase 2b of DSSKP. For these cases, 




—00 if /X > c, 

< 

00 otherwise, 


^ P(r^x > c) = 


1 if /z > c, 

0 otherwise. 


It may be of interest to the reader that it is possible to solve SKPl(W) by 
using the fact that for fixed v = v^x^ > 0 , the objective function in SKPla(W) for 
fixed V is minimized when is maximized. In this case, the objective function of 
SKPl(W) can be solved using the re-arranged model 


p*{W) = mnn^(c - fj,'^x)/y/v), 

where V = { 22 , + 1,..., u} and [u, u] s the range of feasible total variances, t; > 0. 
But this approach is not advisable for solving SKPl(W) because, typically, |V| ll/j. 
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III. A MULTI-STAGE STOCHASTIC 
KNAPSACK PROBLEM 
(MSKP) 


In this chapter, we present an approximate solution to a multi-stage portfolio- 
optimization model using a stochastic dynamic-programming (SDP) approach. This 
multi-stage problem assumes that portfolio revisions can be made at a finite number 
of points evenly spaced in time within a planning horizon of fixed length, i.e., an initial 
portfolio decision is made and rebalanced in stages. Within each stage of this multi¬ 
stage stochastic knapsack problem (MSKP), complete dependence of returns within 
item types is assumed. For returns among item types, there is complete independence 
within each stage and between stages, i.e., we assume complete intra- and inter-stage 
independence among item types. 

A. MATHEMATICAL FORMULATION OF MSKP 

This section presents a formulation of the MSKP using a DP-like recursion. 
In our MSKP, portfolio decisions are made at points in time, t = 0,— 1, and 
the final portfolio is evaluated at t = T. We define the interval of time between t and 
t -h 1 as period t+1. Hence, the portfolio selected at time t is fixed in period t -f 1 
and revised at time t + 1. Variables, parameters and solution sets related to time t 
are labeled with the subscript t; for elements of a vector indexed by t, the index t 
follows after the element index. For example, xu is the fcth element of vector x^. 

The formulation adopts the expected utility maxim (Markowitz 1959, pp. 205- 
242). Utility of a level of wealth indicates the satisfaction of an individual to that level 
of wealth. To maintain continuity with Chapter II, the utility of a realization of wealth 
at time T is defined to be zero unless it meets or exceeds the total return threshold c. 
The utility is one if wealth meets or exceeds the threshold, i.e., the portfolio’s owner 
is fully satisfied with the wealth achieved. This wealth-utility relationship defines 
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the utility function at time T in our problem. A different utility function could be 
specified by the model user without changing our computational framework, although 
a few specialized techniques would be lost or require modification. 

With a given available wealth at time T - 1, we intend to choose a portfolio 
that maximizes the probability of achieving a wealth of at least c at time T. The 
objective function at time T — 1 is, therefore, equivalent to determining a portfolio 
with the mfl.Yirmim expected utility at time T when the portfolio is cashed. Thus, we 
are using the expected utility maxim for our utility function in the last stage. In the 
model described below, this maxim is recursively applied to the portfolio revisions at 
all times t. 

Indices 

t possible transaction times within the planning horizon, t E {0,1,2,..., T} 
Data 

Wt unit cost vector of aU item types at time t 
Wo initial wealth at time 0 

c desired minimum total return, i.e., the total return threshold 

Tt unit return vector of the items at time t for the portfolio selected at time 
t — l; the distribution of r* is multi-variate normal with mean vector and 
variance vector v*; r* = (rit, r 2 t,..., /X 2 t) • • • > v* = 

{vit, V 2 t ,.. •, hence, rkt ~ NitXkt, Vkt) k,t 

ht{W, r^Xt_i) probability density function for realizing wealth W wdth a one-period 
investment in portfolio Xt_i given a random return vector r*; we disallow 
negative portfolio returns so that ht{W, r|’xt_i) = 0 for W <0 

Decision Variables 

Xf portfolio decision at time t, t = 0,1,..., T — 1 

Xt(W) set of feasible portfolio decisions at time t given an available wealth of W, 
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Ut{W) utility of realizing wealth W at time t, t = 1,2,. 


T- 

• ^ 1 


Formulation 


MSKPO 


Uq = max £^fC/i|xol 

xoSA-oCWo) 


( 111 . 1 ) 


s.t. B|(7,|x,_,| = / £/,(iy)A,(W,rfx,_i)<iW' for ( = 1,2. T (111.2) 

Jw 


where Ut{W) = max E^t+i\i^t\ 

xteA’t(w^) 


(111.3) 


for t = 1,2,..., T — 1 and IF > 0 


XtiW) = {xt e Zf |wfxt < W} (in.4) 

for t = 1,2,..., T — 1 and W > 0 

Xo(Wo) = {xo € |wo xo < Wo} . (III.5) 


The intention of the objective function (III.l) is to maximize the expected 
utility by selecting an optimal portfolio decision Xq for period 1 and a suite of optimal 
solution sets (optimal portfolio selection policy) A'* for t = 1,2,...,T — 1. (xg is 
commonly called the first-stage solution or decision.) Given the definition of the 
utility function at time T, the objective value Uq is the probability of achieving or 
exceeding the return threshold c. Constraints (III.2) compute the expected utility, 
at each time t, of portfolio decision Xt_i. Constraints (III.3) show that Ut{W), the 
utility of wealth realization W at time t, is actually the maximum expected utility 
possible at time t + 1. This constraint and the objective function clearly illustrate 
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the recursive resolution of similar sub-problems to determine the final solution for 
the overall problem. The constraints (III.4) and (III.5) define feasible portfolios for 
wealth W at all time t>0, and wealth Wq at time t = 0 respectively. 


B. ASSUMPTIONS AND REFORMULATION 

For the remainder of this chapter, we add the following assumptions to those 
of Chapter II: 


1. The variance of all item types can be integerized through scaling and rounding 
with little loss of accuracy, when necessary. Therefore, we assume that Vk € 
Z+'^keJC. 


2 . 

3. 

4. 


The unit cost and return vectors at different times in the planning horizon are 
identical. Hence, w = wq = wi = • • • = wy, and r = ro = ri = • • • = r^. 
This is for notational simplification only. 

Each riskless item has an unit cost of one, i.e., Wi = l. 

The normal distribution for the total return is left-censored at total return 
W = 0, i.e.. 


r 0 




J-oo y2'KV 




[ ■\/2nv 
where and v — 


ifkr<0, 
iflF = 0, 

otherwise. 


5. The threshold c cannot be achieved or exceeded at the end of the planning 
horizon by making only riskless investments at all stages. 


Now, we formulate an approximation of MSKPO. At each time t except t = 0, 
we discretize the realization of wealth W E [0, oo) into N possible ranges: 


I0.-I = 


(N - l)c 
N 


, c) U [c, oo) 


(IIL6) 


= [w\ W^) u [w^ w®) U • • • U U [ir^, .(III.7) 


Any realized wealth W that falls in the range will be mapped to a spec¬ 

ified value W" in that range. (In this thesis, variables, parameters and solution sets 
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related to the nth realization are labeled with the superscript n.) Of course, the 
choice of VF" will affect the solutions of the model. Solving the problem separately 
with the pessimistic choice W” = fF". and the optimistic choice W” = will 

give, respectively, lower and upper bounds on the optimal solution value to MSKPO. 
(This works even though, strictly speaking, W” = ^ We take a 

“neutral approach” with W” = 0.5(W” + for n = 1,..., — 1 and = c. 

However, in the computational tests, we demonstrate the possible bounds on the op¬ 
timal solution value to MSKPO by using the pessimistic and optimistic choices. The 
discretized formulation is 

Indices 

t possible transaction times within the planning horizon, te {0,1,2,..., T} 
n index for discretized level of wealth, n = {1,2,..., N} 

Data 

w unit cost vector for all item types 
Wo initial wealth at time 0 

c desired minimum total return, i.e., the total return threshold 

[W", W”"''^) nth range of wealth values as defined by equations (III.6) and (III.7) 

P”(x) probability of realizing wealth W G [W”, W”"*"^) from a one-period investment 
in portfolio x; P”(x) = P(W" < r^x < W”"^^) 

W” single representative value for wealth in range [W", n = 1,2,..., iV 

Decision Variables 

Xf portfolio decision at time t, t = 0,1,..., T — 1 

Xp feasible set of portfolio decisions given wealth W” is available for investment 
at time t, t = 1,2,...,T — 1, n = 1,2,..., AT 

17” utility of nth realization of wealth at time t, t = 1,2,..., T—1, n = 1,2,..., AT; 

at time t = Ur = \ 1 

^ 10 otherwise 
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Formulation 


MSKPl 


Uo = max E[Ui\xo] 

xo GA'o 


(III.8) 


N 


s.t. ElUt\x,.i\ = 5;(P"(x,.,)C/rl fort=l,2,...,T (III.9) 


n=l 


where 17" = max jB[17t+i|xt] 


(III.IO) 


for t = 1,2,..., T — 1 and V n 


Xt = {xf e Zffw^xt < W"} (III.ll) 

for t = 1,2,..., T — 1 and V n 

Xo = {xo € |w^xo < Wo} . (III.12) 


We develop a DP-like algorithm to solve the overall problem of MSKPl. In 
the algorithm, the sub-problems of MSKPl, as described by the objective function 
(III.8) and constraints (III.IO), are formulated as mixed-integer programs (MIPs) and 
solved explicitly using a commercial MIP solver. 

C. DYNAMIC PROGRAMMING FOR THE OVERALL 
PROBLEM 

In this section, we present a DP-like algorithm DMSKP to compute the recur¬ 
sion for MSKPl. 

1. Concept 

The recursion in MSKPl can be written in a DP formulation as; 
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DP Recursion for solving MSKPl 
Optimal Value Function 

= maximum expected utility at the time t + 1 given that wealth W" is 
available for investment. 


Recurrence Relation for t = 0,1,..., T — 1, n = 1,2,..., iV 
For t = 

f max E[Ut+i Ixt] for n = 1,2,..., A/’ — 1, 

r/n _ J L ^ I J 

* I 1 for n = iV. 

For t = 0, 


Uq = maxF?[t/i|xo]. 

xo€Ab 

Boundary Conditions 


r 1 ifn = Ar, 

1 0 otherwise. 


Answer 


Uo. 


The resolution of each recursive relation can be handled by an explicit MIP formula¬ 
tion, which will be illustrated in the next section. 

It is well known that the validity of DP is founded on the principle of opti¬ 
mality due to Bellman (any textbook on DP, such as Dreyfus and Law (1977), has 
an explanation of this principle). With respect to the DP formulation above, the 
principle of optimality simply states that if optimal Uo is obtained, then each partial 
solution up obtained must be optimal for its respective state n and stage t, i.e., the 
optimal solution is composed of optimal partial solutions. This principle of optimal¬ 
ity generally requires that the optimal value function be monotonic non-decreasing 
in sequential stages (the monotonicity condition). Hence, conventional DP, which we 
used in DSSKP, requires an additive optimal value function (or additive “accumulated 
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return function” as it is sometimes called) to ensure that monotonicity is satisfied. 
The validity of our proposed DP approach to MSKP holds imder an extension of the 
monotonicity condition. Carraway et al. (1989) state this extended monotonicity 
condition and attribute it to Mitten (1964). 

2. Algorithm DMSKP 

DMSKP is a simple two-phase algorithm. The first phase defines boundary 
conditions, and then carries out a backward recursion from time T — 1 to 0, to 
determine the optimal objective values, C/" (or Uo), and solutions, (or Xq) for 
each realization of wealth PP”. Given t and n, if the deterministic total return from 
a portfolio with only riskless items meets or exceeds c, that portfolio is optimal with 
Uf = 1. Otherwise, 11^ is obtained by solving a MIP. Formulation and solution of 
this MIP is represented by the function EMIP(-) in the algorithm. Phase 2 extracts 
and prints the results. The detailed sequence of DMSKP is: 

Algorithm DMSKP 

Input: Data for MSKPl with K item types: integer vectors /x > 0, v > 0, real vector 
w > 0, scalars Wo > minfeex: Wk, c > muifeeA: Wfc, and {W", n = 1,2,..., AT - 1}. 
Output: Optimal first-stage portfolio decision Xq, and solution sets A/ = {x^*, n = 
1,2...,A^} Vt = l,2,...,T-l. 

Function Called: EMIP(W”, Ut+i, w, /x, v) determines an optimal portfolio decision 
and its utility U” at time t, given available wealth W”, utility function values 
Ut+i = (171^1 ,..., problem data. 

{ 

/* Phase 1 */ 

^1;U^^0V n = 1,2,..., N-1; 

For (t — T — 1 downto 1) { 

For (n = 1 to iV — 1) { 

W' = 0.5(W” -h W”+^); 

If (mi^' > c) 
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Ur ^ 1 ; 

^ 0 V fc # 1; 

else 

^ EMIP(VF”, Ut+i, w, fj,, v); 

} 

} 

4 * - 

X^t* k^l] 

} 

(?7o,xS) ^ EMIP(Wo, Ui, w, fi, v); 

/* Phase 2 */ 

Print {“Solution to MSKP is:”}; 

Print {“t = 0 : C/ = ”, Uo, “x* = ”,xS}; 

For (t = 1 to T — 1 and n = 1 to AT) { 

Print { “t = ”, t, “ : C/ = ”, U^, “x* = ”, x?*}; 

} 

} 

D. EXPLICIT MIXED-INTEGER PROGRAMMING FOR 
THE SUB-PROBLEMS 

This section describes an explicit MIP formulation to solve any sub-problem 
in MSKPl and some proposed pre-processing steps to improve solution efficiency. 

1. Model Formulation 

Suppose a sub-problem requires an optimal portfolio decision at time t as¬ 
suming that wealth W has been achieved firom the investment made at time t — l. 
Extracted from MSKPl, the sub-problem is 
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EUSP 


Uf = maxE[t/f4.i|xt] 

X.t 


S.t. W^Xt 
Xi 


< W' 
e Z^. 


Let the standard normal quantile related to given a portfolio decision x be denoted 
as 

W - /x^x 

y/yT ^2 



where v^x^ ^ 0. Now, denoting the standard normal cumulative distribution as $(•), 
we have, for n = 1,2,..., iV, 


( 4{^=|x) 


P"(x) = $(Z"+'|x) - $(2:”|x) 


if n = 1, 

if n = 2,3,... ,7V — 1, 


1 — $(2^^|xt) otherwise. 


based on the assumption of left-censored normal distribution for the total return. For 
v^xf ^ 0, the objective function of EUSP can, therefore, be re-written as 


■B |Btfi|x<) 


(from (111.9)) 

n=l 

4(^^|x,)(7i, + [0(2"+’|x,) - ^(^"Ix.)] BT+i 

n=2 

+ [i-'i'(2''|x,)]c7'|;, 

n=l 

1 - E since (7^1 = 1. 

71=1 
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If v^xf = 0 and Xf ^ 0, 

E [C/t+i|xt] = Ul\.i such that W” < 

Furthermore, we observe that the expected utility is a direct function of the mean 
and variance of the total portfolio return. Hence, we will also focus on finding the 
best mean-variance combination for the total portfolio return. We present our MIP 
formulation for EUSP: 

Indices and Index Sets 

t index for the time at which an optimal portfolio decision is required 
n index for level of wealth, n € {1,2,..., 

k index for item type, fc € /C = {1,2,..., K} 

i index for mean of total return, i € / 
j index for variance of total return, j E J 

Ji index set of variance values corresponds to a total mean with index i (Note: 

3 = U.^ ■) 

iei 

I index for quantity of an item type 

Ck index set of possible affordable quantities for type-A: item, e {0,1..., L*,} 
where Lk is the maximum number of type-fc items affordable given an available 
wealth at time t 

n'i index for the level of wealth in which a mean total return with index i falls, 
G {1,2,..., N} (see below for further description) 

Data 

p,i the ith level of mean total return, fli —0,1, 

Vj the jth level of total variance in return, = 0,1,..., 

xi quantity of item to invest; xi = I 
fik unit mean return of item type k 
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Vk unit variance in return of item type k 
Wk unit cost of item type k 
W available wealth at time t 

[W^, J^th range of wealth values as defined by equations (III.6) and (111.7) 

(Note: For a portfolio with mean total return jli, denotes n such that 

C/^’Yi optimal utility of achieving wealth level n at time t + 1 
Decision Variables 


Vi 


tj 


binary indicator variable: 

, _ J 1 if portfolio with (fii, Vj) mean-variance combination is selected, 
~ \ 0 otherwise 


l/L 


binary indicator variable: 

1 if units of type-fc item are invested in, 
0 otherwise 


y'ki = 


Formulation 


EMIPI 




I 


N-\ 


y^^y" 


*^2: \ n=l 






V'ii (111.13) 


j€Ji\vj=0 


s.t. Y Y -YY f^kXiy'ia = 0 (III.14) 

i&Jj&Ji k^Kl&Ck 

EE^ (™- 16 ) 

EEs/i = 1 (III17) 

Yvh = IVfce/C (111.18) 

l^Ck 


y'ij ^ {0,1} V i G X, j € Ji 
y'ki ^ {0,1} V A: G Xi, 1 6 £* • 
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Note that if minimum and/or maximum investment levels must be specified for any 
item type, this can be accommodated in EMIPI, and thus in MSKPl, by modifying 
the definition of the set Furthermore, general constraints on the original Xk can 
be added to the model, albeit clumsily. 

The objective function (111.13) maximizes expected utility in EUSP. Although 
the constant 1 in the objective function is unnecessary, it is kept to clarify the link 
between EMIPI and EUSP. Constraints (III.14) and (III.15) ensure that the portfolio 
decision matches the total mean-variance combination selected by the indicator 
variable Constraint (III.16) limits investments by total available wealth. Con¬ 
straint (III.17) specifies that only one total mean-variance combination can be chosen 
as the solution. In a similar manner. Constraint (III. 18) restricts the model to select 
only a single quantity to invest for each item type. 

2. Preprocessing 

If we think of an i-j combination as a grid point, it is clear that the possible 
i-j combinations in EMIPI form a vast search matrix for a MIP solver to find an 
optimal T/ij. This matrix expands pseudo-polynomially with the order of magnitudes 
of the feasible total mean and variance in return; and the solution time of EMIPI 
explodes exponentially with the problem size. Hence, reduction in the size of the 
search matrix is necessary to enable solution of even modest-sized problems. Let 
XJ be the set of candidate i-j combinations for search consideration in EMIPI. We 
suggest the following preprocessing steps to reduce the size of XJ' and prepare data 
before solving EMIPI explicitly with a MIP solver: 

a. Step 1 

First, for fixed wealth W, we wish to determine the set of feasible total 
mean returns. A valid lower bound on feasible total return is defined by 

u = min 
^ fcex 

where mod(a, b) denotes the remainder of a/b. 




Wk 


-t- fj,imod{W',Wk) 
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The upper bound {x on the total mean return is defined by the optimal 
objective value of the knapsack problem KP(M^')- Then, we define index set I = 
{i |For some jli G [fx, jx], 3 x G such that = jxi and w^x = W'}. 

h. Step 2 

Here, we determine for each i G I, a corresponding bounded range for 
total variance, [t;^, Vi\ for a feasible portfolio with pZx = jXi. The corresponding index 
set is denoted hy Ji. A valid lower bound could be found by solving a convex 
non-linear problem: 


VMINO 

• T 2 

Vi = mm v* X 

X 

S.t. fjZx = jXi 

W^x = W' 

X > 0, 

Or, we can add integer restrictions on the x and solving this model which is similar 
in philosophy to EMIPI: 

VMINl 

= rmn E Vfcxfl/w 

^ k^Kl^Ck 
s-t. XI S = jXi 

k^JC> 

S Jl 'f^kXiy'ki = W 

keici^Ck 

Y.^ru = ivie/c 

l€Ck 

t/’i G {0,l}VkeK:,leU 

In practice, we use VMINO rather than VMINl because the tradeoff between compu¬ 
tation speed and bound quality is won by the continuous model in our computational 
tests. 
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There is an exceptional situation when we solve VMINl to get tight 

lower bounds: Note that for fixed i such that hi > c, - ——- < 0 for n = 

1, 2,..., iV — 1 and Vj G [Uj, Uj]. For such an index z, the objective function of EMIPI 


N-\ 

E E 

n-l 






fli 


V^j 


y'ij + E 

jeJi\vj=o 


is maximized when Vj is minimized. Therefore, and we need only 

solve VMINl to find Ji = {u,}. 

In general, an upper bound Vi could be found by solving VMINO with 
the “min” replaced with “max” and the ceiling function replaced by a fioor function. 
This model, called VMAXO, is a difficult non-convex non-linear program, but it is 
known that an optimal solution to VMAXO occurs at an extreme point (Bazaraa 
et al. 1993, pp. 107). Hence, we can find all feasible extreme points of VMAXO 
by enumerating all basic solutions, and evaluating the objective for all basic feasi¬ 
ble solutions. The largest value yields the desired bound. (For the same reason of 
computation speed advantage in the continuous model, we do not solve VMINl with 
“min” replaced by “max.”) 

In particular, for fixed i and W, we try to solve all 2 x 2 systems of 
equations of the form 

/x^x = jli 

w^x = W. 

If such a 2 X 2 system has an unique solution, i.e., its determinant is non-zero, we 
record the solution. If this solution x is feasible, i.e., non-negative, we evaluate the 
total variance v^x^. At the end of the enumeration, the largest total variance found, 
rounded down, is optimal. The algorithm to implement this procedure is: 
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Algorithm MAXVAR 
Input: fi, V, w, W, K and (Li. 

Output: Upper bound Vi on total variance given /t,-. 

{ 

X •«— 0; 

Vi -1; 

For {k = ltoK){ 

For {k' — k+1 to K) { 

det = Wkliy - fikWk'; 

If {det ^ 0){ 

Xk = {W'fik' - iLiWk')/det; 

XkL = {W' - WkXk)/wk'; 

If((a:A.>0) AND > 0)){ 

Vi =max {vi, vl + u^}; 

} 

} 

} 

} 

Vi = 

} 


The converse to the case in which //j > c, has mean wealth in the 
smallest range, i.e., (li < In this situation, the objective function of EMIPI is 
maximized when Vj is maximized. Thus, [uj, Uj] = [uj, Uj] and we can find u,- by solving 
“VMAXl” which is the (unlisted) maximizing version of VMINl. 

c. Step 3 

Here, we compute the standard normal cumulative probability of 
Z‘^\{jli, Vj) for all irj combinations not eliminated in the previous steps. This is a 
simple data preparation step for EMIPI. 
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d. Step 4 

In this step, we determine a lower bound on the optimal objective 
function value and eliminate all i-j combinations that yield worse objective values. 
The lower bound corresponds to a “good” feasible solution to EMIPI found by a 
heuristic solution procedure. First, we compute the objective values of EMIPI for 
all i-j combinations not eliminated in the previous steps. Next, for each item type 
A; > 1, we enumerate all ‘Vealth-consuming” portfolios consisting only of type-A: and 
riskless items, and compute the respective objective values. The maximum objective 
value found is a lower bound for EMIPI. 

e. Sequence of Execution 

The resulting sets XJ obtained from steps 1 and 2, together with the 
cumulative probability values computed in Step 3, are applicable to all sub-problems 
with same initial available wealth W' regardless of time t. Hence, we first carry out 
the preprocessing steps 1, 2 and 3 to generate the set XJ for each value of W = W", 
n = 1,2,,.., iV — 1 before executing DMSKP. For each sub-problem in DMSKP with 
a given available wealth, the algorithm calls the function EMIP(-), which performs 
further reduction of the corresponding input set XJ with preprocessing Step 4 and 
resolution of EMIPI with a MIP solver. This method of running DMSKP with EMIPI 
provides an approximate solution to MSKPO, but an exact solution to the discretized 
problem MSKPl. 

E. COMPUTATIONAL RESULTS 

We test DMSKP on several modest-sized problems here. Problem details are 
listed in Table II. Test Problem 1 is run for three choices of W", specifically, W", 
0.5 (W" + and respectively. 

The algorithm DMSKP with EMIPI is implemented using the General Alge¬ 
braic Modeling System (GAMS) (Brooke et al. 1996). Computations are performed 
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Item Types 

Test Problem 1 

Test Problem 2 

items 1 through 6 
from Table I 

items 1 through 8 
from Table I 

Wq 

30 

30 

C 

80 

80 


20 

10 

VF” V n / AT 

0.5(IU”-1- 

0.5(TU”-t- 


Table II. Details of Test Problems 1 and 2. 

on a Dell Dimension XPS D333 Pentium II computer with 196 megabytes of RAM. 
Computational results for all tests are listed in Table III. 


= 

Test Problem 1 

Test Problem 2 


0.5(IF" + 


0.5(IU” -f 

Uq 

0.9551 

0.9974 

1.0000 

0.9999 

Preprocessing Time 

1.54 

1.92 

1.72 

14.07 

DMSKP Time 

0.87 

9.35 

16.28 

31.97 

Total Time 

2.41 

11.27 

18.00 

46.04 


Table III. Computational results for MSKPl using DMSKP (Times measured in min¬ 
utes.) 


Using Test Problem 1, we compute the lower and upper bounds on the optimal 
objective value of MSKPO by using the pessimistic choice = lU" and optimistic 
choice lU" = respectively. The results are given by the Uq values in Table III. 

Furthermore, the results demonstrate the non-linear relationship between Uq and the 
choice of IF”: Uq for = 0.5(IU" -1-IU"+^) exceeds the average of Uq for = TU" 
and VF" = . 

The solvability of a problem using DMSKP depends very much on problem 
size. In particulaj, solution time using DMSKP grows exponentially with the order 
of magnitude of the unit means and variances of the items, the total return threshold 
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and the number of discretized wealth levels. We compare the results for both test 
problems using the choice W” = 0.5(W" 4- to get some idea of the effect 

of increased problem size on the computational effort. In this case, increasing the 
number of item types by about one third causes a four-fold increase in runtime. So, it 
does appear that computation times can increase significantly with increased number 
of item types. In fact, the increase in problem complexity as items types are added 
depends on the unit weight, and mean and variance in the return of any additional 
item types. In Test Problem 2, it happens that items of type 8 have the maximum 
unit variance among aU item types. Hence, there is a multiplicative increase in the 
size of set XJ. 

For Test Problem 2, there axe 112,906 i-j combinations examined by DMSKP. 
Without preprocessing, this number would be about 900,000, given that p. — 128 and 
V — 7031. It is therefore clear that the preprocessing to reduce the set XJ" is critical 
to the computational efficiency of DMSKP. In addition, further reduction in the size 
of set XJ is possible if the unit variance of the item types have a highest common 
factor a where a 1. In such situations, we can eliminate all i-j combinations with 
mod({;j, o) 7^ 0 and achieve a-fold reduction in the size of XJ, roughly. 
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IV. 


CONCLUSIONS AND FUTURE WORK 


In this thesis, we have developed new methods for solving certain single- and 
multi-stage stochastic knapsack problems (SKPs). The particular problems solved 
are a single-stage integer stochastic portfolio-optimization problem (SPOP) with in¬ 
dependence of returns among the various item types (example, stocks, bonds and 
other financial instruments) in the portfolio, and a multi-stage integer SPOP with 
inter- and intra-stage independence among item types. For both problems, there is 
complete dependence of returns within each item type. Additionally, the return for 
each item type is assumed to follow a normal distribution with known mean and 
variance. Given an avaOable wealth, we wish to determine a portfolio with the best 
probability of achieving or exceeding a specified return threshold at the end of the 
planning horizon. For the single-stage SPOP, portfolio revision is not allowed during 
the planning horizon, whereas for the multi-stage SPOP, periodic portfolio revision 
is allowed. 

An algorithm called DSSKP is developed to solve the single-stage integer 
SPOP using a dynamic-programming (DP) approach. DSSKP is implemented us¬ 
ing the Java programming language. For a problem firom the literature with 11 item 
types, initial wealth of 30 and return threshold of 60, DSSKP obtains an optimal so¬ 
lution in a fraction of a second on a laptop computer. This efficient, exact method is 
easy to program; therefore, it is highly portable to different computer platforms. This 
portability is desirable when DSSKP needs to be run on a more powerful machine 
for a large-scale problem. We also point out that DSSKP can be easily modified to 
accommodate bounded variables, which would be important if quantity restrictions 
were placed on one or more item types. 

For the multi-stage SPOP, possible portfolio returns are discretized into N 
possible levels of wealth for each time period; the nth range is [W”, W""''^), n = 
1,2,..., AT — 1 while the Nth range is [W^, oo). Then, any return or wealth value 
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in [W”, is mapped to a representative value W" chosen from that range. For 

a middle-of-the-road approach, we use IF" = 0.5(1F" -I- At each time period 

but the first, we need to solve N sub-problems to determine an optimal portfolio 
rebalancing decision for each of N levels of wealth. For period 1, we only need to 
solve one sub-problem. The recursive resolution of these sub-problems at each time 
period is handled by a DP-like algorithm, DMSKP. The sub-problems themselves are 
solved with a mixed-integer programming (MIP) model, EMIPI. Using DMSKP with 
EMIPI to solve the discretized multi-stage SPOP exactly provides an approximate 
solution to the original multi-stage SPOP. We obtain lower and upper bounds on 
the optimal objective value of the original problem through the pessimistic choice 
IF" = IF" and optimistic choice W" = IF"'*'^, respectively. 

In this study, DMSKP with EMIPI is implemented using the General Algebraic 
Modeling System (GAMS) (Brooke et al 1996). For a problem with 6 item types, 
initial wealth of 30, wealth interval of 20 (i.e., IF""*"^ — IF" = 20), and final return 
threshold of 80, DMSKP with EMIPI obtains an optimal solution to the discretized 
problem in 11 minutes 17 seconds on a desktop computer. For a larger problem with 
8 item types, initial wealth of 30, wealth interval of 10 and final return threshold of 
80, DMSKP with EMIPI obtains an optimal solution to the discretized problem in 
about 46 minutes. 

In this thesis, we have shown the relevance and efficiency of DP approaches 
for solving a single- and multi-stage SKPs for portfolio optimization. It would be 
more realistic to allow returns to have a general covariance matrix, as in the classical 
Markowitz portfolio-optimization model (Markowitz 1952, 1959). Hence, extensions 
of this thesis should investigate techniques to handle: 

1. Single-stage SKPs with dependencies among item-type returns, 

2. Multi-stage SKPs with intra-stage dependencies among item types, and 

3. Multi-stage SKPs with inter-stage dependencies. 
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The basic model for a single-stage SKP with dependencies among item types 
would look like SKPl(lT’), except that is replaced by x^Vx where V is the co- 
variance matrix of the returns from the various item types. An integer NLP method 
could be applied to solve such a problem. This method could either be a specialized 
algorithm or an explicit model, to be solved using a commercial solver such as DI- 
COPT (GAMS Development Corporation, 1999), which in turn calls a MIP solver 
and an NLP solver. For a problem with a general utility function in the objective 
function, an approximate model would look like EMIPI. Again, the variance compo¬ 
nents would need to be modified appropriately. A firactional programming method 
can be applied to such a problem. But, non-convexities can make it difficult to ensure 
a global optimal solution when the probability of achieving the return threshold is 
less than 1/2. (See Henig 1990 and Geoffrion 1967.) 

Multi-stage SKPs with intra-stage dependencies, but with inter-stage indepen¬ 
dence among item types might be solved in a manner similar to that of Chapter III 
in this thesis. That is, the overall problem would be solved with a DP recursion and 
each sub-problem would be solved by a method developed for a single-stage SKP with 
dependencies among item types. 

For multi-stage SKPs with inter-stage dependencies, a general model is difficult 
to solve for it must handle all kinds of uncertainties, including fluctuation of interest 
rates, changes in the economic health of various business sectors, etc. A limited type 
of inter-stage dependence in the multi-stage SPOP could be handled by redefining 
the state variable; A state would represent a certain range of wealth together with a 
“state of the economy,” or “economy” for short. For each economy, we would have 
a different set of portfolio options available: Industrial stocks might have high mean 
returns in one economy and low returns in another, but agricultural stocks would 
look the same in both scenarios. A Markovian model could govern the transitions 
from economy e at time t to economy e' at time t+1 with known probability Pee't- 
DMSKP and EMIPI can be easily modified to handle such models, at least in theory. 
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One time-dependent parameter that could be handled with ease is the cost 
(weight) of each item type. In this thesis, we assume time-independent deterministic 
costs for all item types for simplicity in notation. These weights can easily be changed 
to time-and-state-dependent parameters. Hence, economy in stage t could affect the 
costs of relevant item types in stage t -fl. 

Unequal interval widths for the wealth ranges might improve the accuracy of 
the discretization. One possibility is to define the interval widths for different wealth 
ranges such that the probability of achieving wealth in the nth level is the same 
for all n. Such an approach would have to be approximated, however, since those 
probabilities depend on the optimal values of the decision variables. 

A possible improvement to the discretization approach for the multi-stage 
SHOP is to view the multi-stage problem as a lattice model and adopt a sampling 
approach. Specifically, importance sampling could be used to reduce the set of feasible 
paths examined in the lattice model to a subset which satisfies certain measures of 
“importance” (Nielsen 1996). 
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